{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.interactiveshell import InteractiveShell\n",
    "InteractiveShell.ast_node_interactivity = 'all'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.读写文本数据\n",
    "\n",
    "使用带有 rt 模式的 open() 函数读取文本文件。\n",
    "\n",
    "使用带有 wt 模式的 open() 函数写入一个文本文件，如果之前文件内容存在则清除并覆盖掉。\n",
    "\n",
    "使用模式为 at 的 open() 函数在已存在文件中添加内容。\n",
    "\n",
    "文件的读写操作默认使用系统编码，可以通过调用 sys.getdefaultencoding() 来得到。\n",
    "\n",
    "open() 有一个 encoding 参数，可以指定读写的文本的编码方式。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "utf-8\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "print(sys.getdefaultencoding())\n",
    "\n",
    "with open('/etc/passwd', 'rt') as f:\n",
    "    data = f.read()\n",
    "    # print(data)\n",
    "    pass\n",
    "\n",
    "with open('/etc/passwd', 'rt') as f:\n",
    "    for line in f:\n",
    "        # print(line)\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.打印输出至文件中\n",
    "\n",
    "在 print() 函数中指定 file 关键字参数，可以将打印内容输出到文件中。\n",
    "\n",
    "文件必须是以文本模式打开。 如果文件是二进制模式的话，打印就会出错。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('/tmp/aaa.txt', 'at') as f:\n",
    "    print('Hello world!', file=f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.使用其他分隔符或行终止符打印\n",
    "\n",
    "你想使用 print() 函数输出数据，但是想改变默认的分隔符或者行尾符。\n",
    "\n",
    "可以使用在 print() 函数中使用 sep 和 end 关键字参数，以你想要的方式输出。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ACME 50 91.5\n",
      "ACME, 50, 91.5\n",
      "ACME, 50, 91.5!!\n"
     ]
    }
   ],
   "source": [
    "print('ACME', 50, 91.5)\n",
    "\n",
    "print('ACME', 50, 91.5, sep=', ')\n",
    "\n",
    "print('ACME', 50, 91.5, sep=', ', end='!!\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "01234"
     ]
    }
   ],
   "source": [
    "for i in range(5):\n",
    "    print(i)\n",
    "\n",
    "for i in range(5):\n",
    "    print(i, end='')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.读写字节数据\n",
    "\n",
    "读写二进制文件，比如图片，声音文件等等。\n",
    "\n",
    "使用模式为 rb 或 wb 的 open() 函数来读取或写入二进制数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'Hello World!'\n"
     ]
    }
   ],
   "source": [
    "with open('/tmp/somefile.bin', 'wb') as f:\n",
    "    f.write(b'Hello World!')\n",
    "\n",
    "with open('/tmp/somefile.bin', 'rb') as f:\n",
    "    data = f.read()\n",
    "    print(data)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.文件不存在才能写入\n",
    "\n",
    "向一个文件中写入数据，但是前提必须是这个文件在文件系统上不存在，也就是不允许覆盖已存在的文件内容。\n",
    "\n",
    "可以在 open() 函数中使用 x 模式来代替 w 模式的方法来解决这个问题。\n",
    "\n",
    "如果文件是二进制的，使用 xb 来代替 xt。\n",
    "\n",
    "> 要注意的是x模式是一个Python3对 open() 函数特有的扩展。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "ename": "FileExistsError",
     "evalue": "[Errno 17] File exists: '/tmp/somefile.bin'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mFileExistsError\u001b[0m                           Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2531756/3941525161.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      3\u001b[0m     \u001b[0mf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwrite\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'12345\\n'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0;32mwith\u001b[0m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfpath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'xt'\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m     \u001b[0mf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwrite\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'abcde\\n'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mFileExistsError\u001b[0m: [Errno 17] File exists: '/tmp/somefile.bin'"
     ]
    }
   ],
   "source": [
    "fpath = '/tmp/somefile.bin'\n",
    "with open(fpath, 'wt') as f:\n",
    "    f.write('12345\\n')\n",
    "\n",
    "with open(fpath, 'xt') as f:\n",
    "    f.write('abcde\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "file already exists\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "fpath = '/tmp/somefile.bin'\n",
    "if not os.path.exists(fpath):\n",
    "    with open(fpath, 'wt') as f:\n",
    "        f.write('Hello\\n')\n",
    "else:\n",
    "    print('file already exists')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.字符串的I/O操作\n",
    "\n",
    "你想使用操作类文件对象的程序来操作文本或二进制字符串。\n",
    "\n",
    "可以使用 io.StringIO() 和 io.BytesIO() 类来创建类文件对象操作字符串数据。\n",
    "\n",
    "io.StringIO 只能用于文本。如果你要操作二进制数据，要使用 io.BytesIO 类来代替。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "'hello world!\\nThis is a test\\n\\n'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "'Hell'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "'o\\n'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "''"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import io\n",
    "\n",
    "s = io.StringIO()\n",
    "s.write('hello world!\\n')\n",
    "print('This is a test\\n', file=s)\n",
    "\n",
    "s.getvalue()\n",
    "\n",
    "s = io.StringIO('Hello\\nWorld\\n')\n",
    "s.read(4)\n",
    "s.read(2)\n",
    "s.read(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7.读写压缩文件\n",
    "\n",
    "你想读写一个gzip或bz2格式的压缩文件。\n",
    "\n",
    "gzip 和 bz2 模块可以很容易的处理这些文件。 两个模块都为 open() 函数提供了另外的实现来解决这个问题。 \n",
    "\n",
    "当写入压缩数据时，可以使用 compresslevel 这个可选的关键字参数来指定一个压缩级别。\n",
    "\n",
    "默认的等级是9，也是最高的压缩等级。等级越低性能越好，但是数据压缩程度也越低。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gzip\n",
    "text = 'hello world'\n",
    "\n",
    "gzpath = '/tmp/somefile.gz'\n",
    "with gzip.open(gzpath, 'wt') as f:\n",
    "    f.write(text)\n",
    "\n",
    "import bz2\n",
    "bzpath = '/tmp/somefile.bz'\n",
    "with bz2.open(bzpath, 'wt') as f:\n",
    "   f.write(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hello world\n",
      "hello world\n"
     ]
    }
   ],
   "source": [
    "import gzip\n",
    "gzpath = '/tmp/somefile.gz'\n",
    "with gzip.open(gzpath, 'rt') as f:\n",
    "    data = f.read()\n",
    "    print(data)\n",
    "\n",
    "import bz2\n",
    "bzpath = '/tmp/somefile.bz'\n",
    "with bz2.open(bzpath, 'rt') as f:\n",
    "    data = f.read()\n",
    "    print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8.固定大小记录的文件迭代\n",
    "\n",
    "你想在一个固定长度记录或者数据块的集合上迭代，而不是在一个文件中一行一行的迭代。\n",
    "\n",
    "可以使用 iter 和 functools.partial() 函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'root:x:0:0:root:/root:/bin/bash\\n'b'daemon:x:1:1:daemon:/usr/sbin:/u'b'sr/sbin/nologin\\nbin:x:2:2:bin:/b'b'in:/usr/sbin/nologin\\nsys:x:3:3:s'b'ys:/dev:/usr/sbin/nologin\\nsync:x'b':4:65534:sync:/bin:/bin/sync\\ngam'b'es:x:5:60:games:/usr/games:/usr/'b'sbin/nologin\\nman:x:6:12:man:/var'b'/cache/man:/usr/sbin/nologin\\nlp:'b'x:7:7:lp:/var/spool/lpd:/usr/sbi'b'n/nologin\\nmail:x:8:8:mail:/var/m'b'ail:/usr/sbin/nologin\\nnews:x:9:9'b':news:/var/spool/news:/usr/sbin/'b'nologin\\nuucp:x:10:10:uucp:/var/s'b'pool/uucp:/usr/sbin/nologin\\nprox'b'y:x:13:13:proxy:/bin:/usr/sbin/n'b'ologin\\nwww-data:x:33:33:www-data'b':/var/www:/usr/sbin/nologin\\nback'b'up:x:34:34:backup:/var/backups:/'b'usr/sbin/nologin\\nlist:x:38:38:Ma'b'iling List Manager:/var/list:/us'b'r/sbin/nologin\\nirc:x:39:39:ircd:'b'/var/run/ircd:/usr/sbin/nologin\\n'b'gnats:x:41:41:Gnats Bug-Reportin'b'g System (admin):/var/lib/gnats:'b'/usr/sbin/nologin\\nnobody:x:65534'b':65534:nobody:/nonexistent:/usr/'b'sbin/nologin\\nsystemd-network:x:1'b'00:102:systemd Network Managemen'b't,,,:/run/systemd:/usr/sbin/nolo'b'gin\\nsystemd-resolve:x:101:103:sy'b'stemd Resolver,,,:/run/systemd:/'b'usr/sbin/nologin\\nsystemd-timesyn'b'c:x:102:104:systemd Time Synchro'b'nization,,,:/run/systemd:/usr/sb'b'in/nologin\\nmessagebus:x:103:106:'b':/nonexistent:/usr/sbin/nologin\\n'b'syslog:x:104:110::/home/syslog:/'b'usr/sbin/nologin\\n_apt:x:105:6553'b'4::/nonexistent:/usr/sbin/nologi'b'n\\ntss:x:106:111:TPM software sta'b'ck,,,:/var/lib/tpm:/bin/false\\nuu'b'idd:x:107:112::/run/uuidd:/usr/s'b'bin/nologin\\ntcpdump:x:108:113::/'b'nonexistent:/usr/sbin/nologin\\nla'b'ndscape:x:109:115::/var/lib/land'b'scape:/usr/sbin/nologin\\npollinat'b'e:x:110:1::/var/cache/pollinate:'b'/bin/false\\nsshd:x:111:65534::/ru'b'n/sshd:/usr/sbin/nologin\\nsystemd'b'-coredump:x:999:999:systemd Core'b' Dumper:/:/usr/sbin/nologin\\nubun'b'tu:x:1000:1000:ubuntu:/home/ubun'b'tu:/bin/bash\\nlxd:x:998:100::/var'b'/snap/lxd/common/lxd:/bin/false\\n'b'ntp:x:112:117::/nonexistent:/usr'b'/sbin/nologin\\n_rpc:x:113:65534::'b'/run/rpcbind:/usr/sbin/nologin\\ns'b'tatd:x:114:65534::/var/lib/nfs:/'b'usr/sbin/nologin\\nbind:x:115:118:'b':/var/cache/bind:/usr/sbin/nolog'b'in\\nlighthouse:x:1001:1001::/home'b'/lighthouse:/bin/bash\\nmysql:x:11'b'6:120:MySQL Server,,,:/nonexiste'b'nt:/bin/false\\ndnsmasq:x:117:6553'b'4:dnsmasq,,,:/var/lib/misc:/usr/'b'sbin/nologin\\n'"
     ]
    }
   ],
   "source": [
    "import functools\n",
    "\n",
    "RECODE_SIZE=32\n",
    "with open('/tmp/somefile.bin', 'rb') as f:\n",
    "    # records 对象是一个可迭代对象，它会不断的产生固定大小的数据块，直到文件末尾。标记值 b'' 就是当到达文件结尾时的返回值。\n",
    "    # functools.partial 用来创建一个每次被调用时从文件中读取固定数目字节的可调用对象。 \n",
    "    records = iter(functools.partial(f.read, RECODE_SIZE), b'')\n",
    "    for r in records:\n",
    "        print(r, end='')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9.读取二进制数据到可变缓冲区中\n",
    "\n",
    "你想直接读取二进制数据到一个可变缓冲区中，而不需要做任何的中间复制操作。 或者你想原地修改数据并将它写回到一个文件中去。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bytearray(b'root:x:0:0:root:/root:/bin/bash\\ndaemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\\nbin:x:2:2:bin:/bin:/usr/sbin/nologin\\nsys:x:3:3:sys:/dev:/usr/sbin/nologin\\nsync:x:4:65534:sync:/bin:/bin/sync\\ngames:x:5:60:games:/usr/games:/usr/sbin/nologin\\nman:x:6:12:man:/var/cache/man:/usr/sbin/nologin\\nlp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\\nmail:x:8:8:mail:/var/mail:/usr/sbin/nologin\\nnews:x:9:9:news:/var/spool/news:/usr/sbin/nologin\\nuucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\\nproxy:x:13:13:proxy:/bin:/usr/sbin/nologin\\nwww-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\\nbackup:x:34:34:backup:/var/backups:/usr/sbin/nologin\\nlist:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\\nirc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\\ngnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\\nnobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\\nsystemd-network:x:100:102:systemd Network Management,,,:/run/systemd:/usr/sbin/nologin\\nsystemd-resolve:x:101:103:systemd Resolver,,,:/run/systemd:/usr/sbin/nologin\\nsystemd-timesync:x:102:104:systemd Time Synchronization,,,:/run/systemd:/usr/sbin/nologin\\nmessagebus:x:103:106::/nonexistent:/usr/sbin/nologin\\nsyslog:x:104:110::/home/syslog:/usr/sbin/nologin\\n_apt:x:105:65534::/nonexistent:/usr/sbin/nologin\\ntss:x:106:111:TPM software stack,,,:/var/lib/tpm:/bin/false\\nuuidd:x:107:112::/run/uuidd:/usr/sbin/nologin\\ntcpdump:x:108:113::/nonexistent:/usr/sbin/nologin\\nlandscape:x:109:115::/var/lib/landscape:/usr/sbin/nologin\\npollinate:x:110:1::/var/cache/pollinate:/bin/false\\nsshd:x:111:65534::/run/sshd:/usr/sbin/nologin\\nsystemd-coredump:x:999:999:systemd Core Dumper:/:/usr/sbin/nologin\\nubuntu:x:1000:1000:ubuntu:/home/ubuntu:/bin/bash\\nlxd:x:998:100::/var/snap/lxd/common/lxd:/bin/false\\nntp:x:112:117::/nonexistent:/usr/sbin/nologin\\n_rpc:x:113:65534::/run/rpcbind:/usr/sbin/nologin\\nstatd:x:114:65534::/var/lib/nfs:/usr/sbin/nologin\\nbind:x:115:118::/var/cache/bind:/usr/sbin/nologin\\nlighthouse:x:1001:1001::/home/lighthouse:/bin/bash\\nmysql:x:116:120:MySQL Server,,,:/nonexistent:/bin/false\\ndnsmasq:x:117:65534:dnsmasq,,,:/var/lib/misc:/usr/sbin/nologin\\n')\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "def read_info_buff(filename):\n",
    "    buf = bytearray(os.path.getsize(filename))\n",
    "    with open(filename, 'rb') as f:\n",
    "        f.readinto(buf)\n",
    "    return buf\n",
    "\n",
    "retBuf = read_info_buff('/tmp/somefile.bin')\n",
    "print(retBuf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 10.内存映射的二进制文件\n",
    "\n",
    "你想内存映射一个二进制文件到一个可变字节数组中，目的可能是为了随机访问它的内容或者是原地做些修改。\n",
    "\n",
    "可以使用 mmap 模块来内存映射文件。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2125\n",
      "b'root:x:0:0'\n"
     ]
    }
   ],
   "source": [
    "import os, mmap\n",
    "\n",
    "def memory_map(filename, access=mmap.ACCESS_WRITE):\n",
    "    size = os.path.getsize(filename)\n",
    "    fd = os.open(filename, os.O_RDWR)\n",
    "    return mmap.mmap(fd, size, access)\n",
    "\n",
    "with memory_map('/tmp/somefile.bin') as m:\n",
    "    print(len(m))\n",
    "    print(m[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 11.文件路径名的操作\n",
    "\n",
    "你需要使用路径名来获取文件名，目录名，绝对路径等等。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'odoo_15.0.20220410.zip'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "'/home/ubuntu/tmps'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "'tmp/data/odoo_15.0.20220410.zip'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "'/home/ubuntu/testdir'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "('/home/ubuntu/tmps', 'odoo_15.0.20220410.zip')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "('/home/ubuntu/tmps/odoo_15.0.20220410', '.zip')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "path = '/home/ubuntu/tmps/odoo_15.0.20220410.zip'\n",
    "\n",
    "# Get the last component of the path\n",
    "os.path.basename(path)\n",
    "\n",
    "# Get the directory name\n",
    "os.path.dirname(path)\n",
    "\n",
    "os.path.join('tmp', 'data', os.path.basename(path))\n",
    "\n",
    "os.path.expanduser('~/testdir')\n",
    "\n",
    "os.path.split(path)\n",
    "os.path.splitext(path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.测试文件是否存在"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "os.path.exists('/etc/passwd')\n",
    "os.path.exists('/tmp/swapn')\n",
    "\n",
    "print('-'*20)\n",
    "os.path.isfile('/etc/passwd')\n",
    "os.path.isdir('/etc/passwd')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 13.获取文件夹中的文件列表"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['wwwroot', 'IoT-Beginner', 'ubuntu', 'lighthouse']"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['wwwroot', 'IoT-Beginner', 'ubuntu', 'lighthouse']\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "os.listdir('/home')\n",
    "\n",
    "# Get all dirs\n",
    "dirnames = [name for name in os.listdir('/home') if os.path.isdir(os.path.join('/home', name))]\n",
    "print(dirnames)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['/home/ubuntu']"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 对于文件名的匹配，你可能会考虑使用 glob 或 fnmatch 模块。\n",
    "import glob\n",
    "glob.glob('/home/ubun*')\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 14.忽略文件名编码\n",
    "\n",
    "默认情况下，所有的文件名都会根据 sys.getfilesystemencoding() 返回的文本编码来编码或解码。\n",
    "\n",
    "你想使用原始文件名执行文件的I/O操作，也就是说文件名并没有经过系统默认编码去解码或编码过。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 15.打印不合法的文件名\n",
    "\n",
    "你的程序获取了一个目录中的文件名列表，但是当它试着去打印文件名的时候程序崩溃， 出现了 UnicodeEncodeError 异常和一条奇怪的消息—— surrogates not allowed 。\n",
    "\n",
    "默认情况下，Python假定所有文件名都已经根据 sys.getfilesystemencoding() 的值编码过了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/tmp/somefile.bin\n"
     ]
    }
   ],
   "source": [
    "# 当打印未知的文件名时，使用下面的方法可以避免这样的错误：\n",
    "def bad_filename(filename):\n",
    "    return repr(filename)[1:-1]\n",
    "\n",
    "filename = '/tmp/somefile.bin'\n",
    "try:\n",
    "    print(filename)\n",
    "except UnicodeEncodeError:\n",
    "    print(bad_filename(filename))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 16.增加或改变已打开文件的编码\n",
    "\n",
    "如果你想给一个以二进制模式打开的文件添加Unicode编码/解码方式， 可以使用 io.TextIOWrapper() 对象包装它。\n",
    "\n",
    "I/O系统由一系列的层次构建而成。你可以试着运行下面这个操作一个文本文件的例子来查看这种层次。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_io.TextIOWrapper name='/tmp/somefile.bin' mode='w' encoding='UTF-8'>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<_io.BufferedWriter name='/tmp/somefile.bin'>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<_io.FileIO name='/tmp/somefile.bin' mode='wb' closefd=True>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f = open('/tmp/somefile.bin', 'w')\n",
    "f\n",
    "f.buffer\n",
    "f.buffer.raw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "io.TextIOWrapper 是一个编码和解码Unicode的文本处理层， io.BufferedWriter 是一个处理二进制数据的带缓冲的I/O层， io.FileIO 是一个表示操作系统底层文件描述符的原始文件。\n",
    "\n",
    "增加或改变文本编码会涉及增加或改变最上面的 io.TextIOWrapper 层。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_io.TextIOWrapper name='/tmp/somefile.bin' mode='w' encoding='UTF-8'>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<_io.TextIOWrapper name='/tmp/somefile.bin' encoding='latin-1'>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sys\n",
    "import io\n",
    "f = open('/tmp/somefile.bin', 'w')\n",
    "f\n",
    "\n",
    "# detach() 方法会断开文件的最顶层并返回第二层，之后最顶层就没什么用了。\n",
    "f = io.TextIOWrapper(f.detach(), encoding='latin-1')\n",
    "f"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 17.将字节写入文本文件\n",
    "\n",
    "将字节数据直接写入文件的缓冲区即可。\n",
    "\n",
    "类似的，能够通过读取文本文件的 buffer 属性来读取二进制数据。\n",
    "\n",
    "I/O系统以层级结构的形式构建而成。 文本文件是通过在一个拥有缓冲的二进制模式文件上增加一个Unicode编码/解码层来创建。 buffer 属性指向对应的底层文件。如果你直接访问它的话就会绕过文本编码/解码层。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "with open('/tmp/somefile.bin', 'w') as f:\n",
    "    f.buffer.write(b'Hello !!\\n')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 18.将文件描述符包装成文件对象"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "fd = os.open('/tmp/somefile.bin', os.O_WRONLY | os.O_CREAT)\n",
    "\n",
    "f = open(fd, 'wt', closefd=False)\n",
    "f.write('aaaaaa\\n')\n",
    "f.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 19.创建临时文件和文件夹\n",
    "\n",
    "你需要在程序执行时创建一个临时文件或目录，并希望使用完之后可以自动销毁掉。\n",
    "\n",
    "tempfile 模块中有很多的函数可以完成这任务。\n",
    "\n",
    "- TemporaryFile()\n",
    "- NamedTemporaryFile()\n",
    "- TemporaryDirectory()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello World \n",
      "Testng\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import tempfile\n",
    "\n",
    "with tempfile.TemporaryFile('w+t') as f:\n",
    "    f.write('Hello World \\n')\n",
    "    f.write('Testng\\n')\n",
    "\n",
    "    f.seek(0)\n",
    "    data = f.read()\n",
    "    print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.与串行端口的数据通信\n",
    "\n",
    "尽管你可以通过使用Python内置的I/O模块来完成这个任务，但对于串行通信最好的选择是使用 pySerial包 。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 21.序列化Python对象\n",
    "\n",
    "对于序列化最普遍的做法就是使用 pickle 模块。\n",
    "\n",
    "一个对象转储为一个字符串，可以使用 pickle.dump(), pickle.dumps()。\n",
    "\n",
    "从字节流中恢复一个对象，使用 pickle.load() 或 pickle.loads() 函数。\n",
    "\n",
    "由于 pickle 是Python特有的并且附着在源码上，所有如果需要长期存储数据的时候不应该选用它。 例如，如果源码变动了，你所有的存储数据可能会被破坏并且变得不可读取。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 最简单的示例\n",
    "import pickle\n",
    "\n",
    "data  = [1, 2, 3]\n",
    "with open('/tmp/somefile.bin', 'wb') as f:\n",
    "    pickle.dump(data, f)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.7 ('base')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "caf1c2fcf97217de91eafa76b907d50f9ea378f5ffbee7f571142d119bb6a771"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
